Packet forwarding method and related apparatus

ABSTRACT

This application provides a packet forwarding method and a related apparatus. The method includes: determining whether a flow to which a packet belongs meets a large flow condition; and if the flow to which the packet belongs meets the large flow condition and a flowlet routing table does not include a flowlet entry corresponding to the flow, creating the corresponding flowlet entry in the flowlet routing table, and forwarding the packet based on the created flowlet entry. In the foregoing technical solution, a corresponding flowlet entry may be created only for a large flow, to perform flowlet-based load balancing on the large flow. This can reduce an entry resource used for flowlet-based load balancing, improve balance of multi-path load sharing, and effectively improve a traffic sharing effect of load balancing.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chines Patent Application No.202010975328.0, filed on Sep. 16, 2020, which is hereby incorporated byreference in its entirety.

TECHNICAL FIELD

This application relates to the field of communications technologies,and more specifically, to a packet forwarding method and a relatedapparatus.

BACKGROUND

In a data communication network, a plurality of switches and routers(hereinafter collectively referred to as forwarding nodes) form anetwork for forwarding network traffic. A single data packet (forexample, a layer 3 internet protocol (IP) packet) in the network trafficmay be routed to a destination network device (for example, a computer)by using the forwarding nodes through different paths. When a largeamount of network traffic passes through a forwarding node, a layer 3load balancing technology is used for the forwarding node to reducenetwork congestion and a packet forwarding delay. When a same flow has aplurality of forwarding paths, the forwarding node having the layer 3layer 3 load balancing technology selects to forward all or some packetsof the flow to different paths, to implement traffic balancing on thedifferent paths.

Equal-cost multi-path routing (ECMP) is used in a conventional layer 3load balancing technology. A forwarding node using the ECMP stores anECMP routing table. After receiving a data packet, the forwarding nodecalculates an index based on flow identity information of the datapacket, then queries the ECMP routing table for an entry matching theindex, and forwards the data packet based on a next-hop port included inthe entry. However, the ECMP has a poor load balancing effect in somescenarios.

SUMMARY

This application provides a packet forwarding method and a relatedapparatus, to improve balance of multi-path load sharing.

An embodiment of this application provides a packet forwarding method,including: determining a first identifier based on flow identityinformation of a first packet, where the first identifier is anidentifier of the first packet; and when a first flow to which the firstpacket belongs is a large flow and a flowlet routing table does notinclude a first flowlet entry corresponding to the first flow, creatingthe first flowlet entry in the flowlet routing table, and forwarding thefirst packet based on the first flowlet entry, where the first flowletentry includes the first identifier.

In the foregoing technical solution, a corresponding flowlet entry maybe created only for a large flow, to perform flowlet-based loadbalancing on the large flow. This can reduce an entry resource used forflowlet-based load balancing, improve balance of multi-path loadsharing, and effectively improve a traffic sharing effect of loadbalancing.

In an embodiment, the flowlet routing table includes at least oneflowlet entry, the at least one flowlet entry is in a one-to-onecorrespondence with at least one flow, and the first flow is any one ofthe at least one flow. Because each flowlet entry corresponds to onlyone flow, a problem that effective load balancing cannot be implementedto share traffic because a plurality of flows match one flowlet entrycan be reduced.

In an embodiment, the at least one flowlet entry is a valid entry in theflowlet routing table.

In an embodiment, the creating the first flowlet entry includes:determining a first dynamic entry from a dynamic routing table based onthe first identifier, where an identifier field of the first dynamicentry includes the first identifier; and determining a first next-hopport in a plurality of next-hop ports included in the first dynamicentry, where the first next-hop port is a next-hop port included in anext-hop port field in the first flowlet entry.

In an embodiment, the determining a first next-hop port in a pluralityof next-hop ports included in the first dynamic entry includes:obtaining a plurality of pieces of path quality, where the plurality ofpieces of path quality are in a one-to-one correspondence with theplurality of next-hop ports; and determining a next-hop portcorresponding to a piece of path quality that meets a preset conditionin the plurality of pieces of path quality as the first next-hop port.

In an embodiment, the method further includes: determining a secondidentifier based on flow identity information of a second packet, wherethe second identifier is an identifier of the second packet; and when asecond flow to which the second packet belongs is not a large flow,forwarding the second packet based on the second identifier and anequal-cost multi-path routing ECMP routing table.

In the foregoing technical solution, an ECMP technology is used toforward a flow that is not a large flow. In other words, whether toforward a packet based on the flowlet routing table or the ECMP routingtable may be determined based on whether a flow to which the packetbelongs is a large flow. This can improve the balance of multi-path loadsharing, and effectively improved the traffic sharing effect of loadbalancing.

In an embodiment, the method further includes: receiving a third packet;determining a third identifier based on flow identity information of thethird packet, where the third identifier is an identifier of the thirdpacket; determining, based on the third identifier, whether the flowletrouting table includes a third flowlet entry, where the third flowletentry includes the third identifier; and if the flowlet routing tableincludes the third flowlet entry, forwarding the third packet based onthe third flowlet entry, or if the flowlet routing table does notinclude the third flowlet entry, determining whether a third flow towhich the third packet belongs is a large flow, and based on adetermined result, creating the third flowlet entry in the flowletrouting table or forwarding the third packet based on the ECMP routingtable.

In an embodiment, the method further includes: receiving a fourthpacket; determining a fourth identifier based on flow identityinformation of the fourth packet, where the fourth identifier is anidentifier of the fourth packet; determining whether a fourth flow towhich the fourth packet belongs is a large flow; and when the fourthflow is a large flow, forwarding the fourth packet based on a fourthflowlet entry in the flowlet routing table, where the fourth flowletentry includes the fourth identifier, or when the fourth flow is not alarge flow, forwarding the fourth packet based on the ECMP routingtable.

According to a second aspect, an embodiment of this application furtherprovides a network device. The network device includes modulesconfigured to implement the method according to any one of the firstaspect or the possible designs of the first aspect.

According to a third aspect, an embodiment of this application furtherprovides a network device, including a processor. The processor isconfigured to be coupled to a memory, and read and execute instructionsand/or program code in the memory, to perform the method according toany one of the first aspect or the possible designs of the first aspect.

According to a fourth aspect, an embodiment of this application furtherprovides a chip system, including a logic circuit. The logic circuit isconfigured to be coupled to an input/output interface, and transmit datathrough the input/output interface, to perform the method according toany one of the first aspect or the possible designs of the first aspect.

According to a fifth aspect, an embodiment of this application furtherprovides a computer-readable storage medium. The computer-readablemedium stores program code, and when the computer program code is run ona network device, the network device is enabled to perform the methodaccording to any one of the first aspect or the possible designs of thefirst aspect.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic structural diagram of a forwarding node accordingto an embodiment of this application;

FIG. 2 is a flowchart of packet processing corresponding to a processingmanner 1;

FIG. 3 is another schematic structural diagram of a forwarding node;

FIG. 4 is a flowchart of packet processing corresponding to a processingmanner 2;

FIG. 5 is another schematic structural diagram of a forwarding node.

DESCRIPTION OF EMBODIMENTS

The following describes the technical solutions in this application withreference to the accompanying drawings.

All aspects, embodiments, or features are presented in this applicationby describing a system that may include a plurality of devices,components, modules, and the like. It should be appreciated andunderstood that, each system may include another device, component,module, and the like, and/or may not include all devices, components,modules, and the like discussed with reference to the accompanyingdrawings. In addition, a combination of these solutions may be used.

In addition, in the embodiments of this application, the terms such as“example” and “such as” are used to represent giving an example, anillustration, or a description. Any embodiment or design schemedescribed as an “example” in this application should not be explained asbeing more preferred or having more advantages than another embodimentor design scheme. Exactly, “for example” is used to present a concept ina specific manner.

In the embodiments of this application, “corresponding (corresponding,relevant)” and “corresponding” may be interchangeably used sometimes. Itshould be noted that meanings expressed by the terms are consistent whendifferences are not emphasized.

In the embodiments of this application, sometimes a subscript such as W₁may be written in an incorrect form such as W1. Expressed meanings areconsistent when differences between them are not emphasized.

A network architecture and a service scenario described in theembodiments of this application are intended to describe the technicalsolutions in the embodiments of this application more clearly, and donot constitute a limitation on the technical solutions provided in theembodiments of this application. A person of ordinary skill in the artmay know that, with evolution of the network architecture and emergenceof new service scenarios, the technical solutions provided in theembodiments of this application are also applicable to similar technicalproblems.

Reference to “an embodiment”, “some embodiments”, or the like describedin this specification indicates that one or more embodiments of thisapplication include a specific feature, structure, or characteristicdescribed with reference to the embodiments. Therefore, in thisspecification, statements, such as “in an embodiment”, “in someembodiments”, “in some other embodiments”, and “in other embodiments”,that appear at different places do not necessarily mean referring to asame embodiment, instead, the statements mean referring to “one or morebut not all of the embodiments”, unless otherwise specificallyemphasized in other ways. The terms “include”, “comprise”, “have”, andvariants of the terms all mean “include but are not limited to”, unlessotherwise specifically emphasized in other ways.

In this application, “at least one” means one or more, and “a pluralityof” means two or more. The term “and/or” describes an associationrelationship between associated objects and represents that threerelationships may exist. For example, A and/or B may represent thefollowing cases: Only A exists, both A and B exist, and only B exists,where A and B may be singular or plural. The character “/” usuallyrepresents an “or” relationship between the associated objects. “Atleast one item (piece) of the following” or a similar expression thereofmeans any combination of these items, including a singular item (piece)or any combination of plural items (pieces). For example, at least oneitem (piece) of a, b, or c may represent: a, b, c, a and b, a and c, band c, or a, b, and c, where a, b, and c may be singular or plural.

Embodiments of this application provide a packet forwarding method. Themethod may be performed by a forwarding node or a component (forexample, a chip or a circuit) in the forwarding node. For ease ofdescription, in the following embodiments, the technical solutions inthis application are described by using the forwarding node as anexecution body.

The forwarding node may forward a packet based on a flowlet routingtable or an ECMP routing table. Each flowlet entry in the flowletrouting table is created when a corresponding flow meets a large flowcondition. Therefore, a flow to which a packet forwarded based on theflowlet routing table belongs meets the large flow condition. A flow towhich a packet forwarded based on the ECMP routing table belongs doesnot meet the large flow condition. In other words, according to themethod provided in the embodiments of this application, whether toforward a packet based on the flowlet routing table or the ECMP routingtable may be determined based on whether a flow to which the packetbelongs meets the large flow condition.

In network traffic, a flow is a group of packets having a same flowidentifier (for example, a quintuple or a flow label). A large flowrefers to a flow in which a quantity of packets exceeds a threshold. Ina specific implementation, a group of packets in a flow in which aquantity of packets exceeds the threshold within a period of time isidentified as a large flow. A flow meeting the large flow condition maybe considered as a large flow, or may be considered as a large burst.Correspondingly, a flow that does not meet the large flow condition maybe considered as a small flow, or may be considered as a small burst.The large flow may also be referred to as an elephant flow, and thesmall flow may also be referred to as a mice flow.

In the foregoing technical solution, flowlet-based load balancing may beperformed only on a large flow that meets the large flow condition, andECMP may be used to perform load balancing on a small flow that does notmeet the large flow condition. This can reduce an entry resource usedfor flowlet-based load balancing, improve balance of multi-path loadsharing, and effectively improve a traffic sharing effect of loadbalancing.

As described above, a flowlet entry in the flowlet routing table iscreated only when a flow to which a packet belongs meets the large flowcondition. Therefore, in an initial phase (that is, when the forwardingnode has just been deployed on a network and has not received anypacket), the flowlet routing table is an empty routing table. As aquantity of packets received by the forwarding node increases, theforwarding node may determine a flow that meets the large flowcondition, and create a corresponding flowlet entry in the flowletrouting table. In this way, after a new packet in a flow for which acorresponding flowlet entry has been created arrives at the forwardingnode, how to process the packet may be determined based on the matchedflowlet entry in the flowlet routing table.

FIG. 1 is a schematic structural diagram of a forwarding node accordingto an embodiment of this application. As shown in FIG. 1 , theforwarding node 100 includes an identifier calculation module 110, aflowlet query module 120, a determining module 130, a flowlet routingassignment module 140, a flowlet routing forward module 150, and an ECMProuting forward module 160.

The following separately describes functions of the identifiercalculation module 110, the flowlet query module 120, the determiningmodule 130, the flowlet routing assignment module 140, the flowletrouting forward module 150, and the ECMP routing forward module 160.

(1) The Identifier Calculation Module 110

The identifier calculation module 110 is configured to calculate anidentifier of a packet. Each time the forwarding node receives a packet,the identifier calculation module 110 calculates an identifier of thepacket. For example, the identifier calculation module 110 may determinean identifier of a packet based on flow identity information of thepacket. Flow identity information is information used to identify anidentity of a flow. Flow identity information of different flows isdifferent, and flow identity information of different packets in a sameflow is the same.

The flow identity information may also be referred to as a flowidentifier (flow ID).

For an internet protocol version 4 (IPv4) packet, flow identityinformation of the IPv4 packet may be a quintuple, to be specific, asource IP address, a destination IP address, a source port, adestination port, and a protocol type that are carried in a packetheader. In some other embodiments, the flow identity information of theIPv4 packet may alternatively be a source IP address and a destinationIP address. In some other embodiments, the flow identity information ofthe IPv4 packet may alternatively be a source IP address, a destinationIP address, a source port, and a destination port.

For an internet protocol version 6 (IPv6) packet, flow identityinformation of the IPv6 packet may be a flow label. In some otherembodiments, the flow identity information of the IPv6 packet mayfurther include a source address and a destination address in additionto the flow label.

The identifier calculation module 110 may perform a hash operation onflow identity information of a packet to obtain an identifier of thepacket. The identifier of the packet may be a hash value obtained byperforming the hash operation on the flow identity information of thepacket. An algorithm that may be used to calculate the hash value of theflow identity information of the packet may be a cyclic redundancy check32 (CRC32), a message-digest algorithm 5 (MD5), a secure hash algorithm(SHA) (for example, SHA-0, SHA-2, or SHA-256), or the like. This is notlimited in the embodiments of this application.

(2) The Flowlet Query Module 120

The flowlet query module 120 stores a flowlet routing table. Eachflowlet entry in the flowlet routing table mainly includes the followingfields: an identifier field, a valid information field, and a next-hopport field.

The identifier field in the flowlet entry includes an identifier. If anidentifier of a packet is the same as an identifier included in anidentifier field in a flowlet entry, it may be considered that theidentifier matches the flowlet entry. In this case, how to process thepacket may be further determined based on a valid information field anda next-hop port field in the matched flowlet entry.

In some embodiments, different flow identity information determinesdifferent identifiers. In this case, the flowlet entry in the flowletrouting table is in a one-to-one correspondence with a flow. In otherwords, different flowlet entries correspond to different flows, and aquantity of flowlet entries included in the flowlet routing table is thesame as a quantity of matched flows.

In some other embodiments, different flow identity information maydetermine a same identifier. In this case, one flowlet entry maycorrespond to a plurality of flowlet flows. In other words, a quantityof flowlet entries included in the flowlet routing table may be lessthan a quantity of matched flows.

The valid information field in the flowlet entry is used to determinewhether a matched packet can be forwarded through a next-hop portrecorded in the next-hop port field in the flowlet entry.

In some embodiments, the valid information field may include a valid bitfield and an aging flag field.

In some other embodiments, the valid information field may include avalid bit field and a timestamp field.

A value of the valid bit field may be 1 or 0.

If the value of the valid bit field in the flowlet entry is 1, itindicates that the flowlet entry is a valid routing entry. In this case,if the packet does not time out, the matched packet may be forwardedthrough the next-hop port recorded in the next-hop port field in theflowlet entry.

If the value of the valid bit field in the flowlet entry is 0, itindicates that the flowlet entry is not a valid routing entry. In thiscase, the flowlet entry cannot be used as a routing entry used toprocess the packet matching the flowlet entry. In this case, the flowletrouting assignment module 140 needs to re-assign a new next-hop port forthe packet and set the valid bit field to 1.

In some embodiments, if the value of the valid bit field in the flowletentry is 0, it indicates that the flowlet entry is a valid routingentry. If the value of the valid bit field in the flowlet entry is 1, itindicates that the flowlet entry is not a valid routing entry.

A value of the aging flag field may be 1 or 0. The aging flag field maybe used to determine whether a flowlet to which a packet belongs isaged.

The flowlet query module 120 may maintain a timer. A time threshold Tmay be set for the timer. The time threshold T is a minimum interval fordetermining whether two packets belong to a same flowlet. The value ofthe aging flag field is determined based on whether an interval at whichtwo packets are processed is greater than the time threshold T. When avalid bit is 1, an aging bit is set to 1 when any packet arrives. Whenthe timer expires, if the aging bit is 0, it may be considered that aflowlet to which the packet belongs is aged. In this case, the valid bitis set to 0. If the aging bit is 1, the valid bit remains unchanged. Theaging bit is always set to 0 after the timer expires.

It is assumed that a packet 1 and a packet 2 are two consecutive packetsin a same flow received by the forwarding node 100, and time at whichthe packet 1 is processed is t1. It is assumed that the packet 1 is aprevious packet of the packet 2. The flowlet query module 120 may startthe timer at t1. t1. If the timer exceeds the time threshold T and thepacket 2 is not processed, it may be considered that a flowlet to whichthe packet 2 belongs is aged. In this case, the value of the aging flagfield and the value of the valid bit field in the flowlet entry may beset to 0. If the packet 2 has been processed before the timer exceedsthe time threshold T, it may be considered that the flowlet to which thepacket 2 belongs is not aged. In this case, the aging flag field in theflowlet entry is 1. The timer continues to count. When the timer reachesthe time threshold T, the aging flag field in the flowlet entry is setto 0.

Processing time of a packet may be time at which the forwarding node 100receives the packet, or time at which the identifier calculation module110 obtains an identifier of the packet through calculation, or time atwhich the flowlet query module 120 obtains the packet. This is notlimited in the embodiments of this application.

The timestamp field is used to record processing time of a previouspacket. The timestamp field may be used to determine whether a flowletto which a packet belongs is aged. When a valid bit is 1, processingtime of a previous packet of a current packet recorded in the timestampfield may be subtracted from processing time of the current packet. If asubtraction result is greater than a preset time threshold, it may beconsidered that a flowlet to which the current packet belongs is aged,and the valid bit is set to 0. It indicates that the flowlet is nolonger valid. If the subtraction result is less than or equal to thepreset time threshold, it may be considered that the flowlet to whichthe current packet belongs is not aged, and a value of the valid bitremains 1.

The packet 1 and the packet 2 are further used as an example. It isassumed that T is still used to represent the time threshold, and t2 isused to represent processing time of the packet 2. The timestamp fieldrecords the processing time of the packet 1, namely, t1. It is assumedthat the value of the valid bit is 1. If t2−t1>T, the valid bit is setto 0. If t2−t1≤T, the value of the valid bit remains unchanged.

In conclusion, the value of the valid bit of the flowlet entry may bedetermined based on the value of the aging flag field or the value ofthe timestamp field. When the value of the valid bit field is 1, if itis determined, based on the aging flag field or the timestamp field,that the flowlet to which the packet belongs is aged, the value of thevalid bit field may be set to 0. If it is determined, based on the agingflag field or the timestamp field, that the flowlet to which the packetbelongs is not aged, the value of the valid bit field may remainunchanged.

If the value of the valid bit field is 0, it may be considered that thecurrent packet and the previous packet of the current packet belong todifferent flowlets. For example, the current packet may be a firstpacket of a next flowlet of the flow, or a first packet of a flowlet ofa flow with a same identifier.

(3) The Determining Module 130

The determining module 130 is configured to determine whether a flow towhich a packet belongs meets a large flow condition.

In some embodiments, the determining module 130 may determine, by usinga sliding window algorithm, whether a flow to which a packet belongsmeets the large flow condition.

In some embodiments, the determining module 130 may store a slidingwindow counting bucket. The sliding window counting bucket may includeat least one hash table. Each of the at least one hash table includes aplurality of statistical entries. Each of the plurality of statisticalentries may include the following fields: an identifier field, a countvalue field, and a large flow flag field. The determining module 130 mayobtain an identifier of a packet determined by the identifiercalculation module 110, and determine whether a statistical entry(hereinafter referred to as a matched statistical entry) matching thepacket exists in each of the at least one hash table. An identifier inan identifier field of the statistical entry matching the packet is thesame as the identifier of the packet.

If only one hash table in the at least one hash table includes a matchedstatistical entry, a count value stored in a count value field of thematched statistical entry may be increased by 1, and then an updatedcount value (namely, the count value increased by 1) is compared with apreset threshold. If the updated count value is greater than the presetthreshold, it may be determined that the flow to which the packetbelongs meets the large flow condition. If the updated count value isnot greater than the preset threshold, it may be determined that theflow to which the packet belongs does not meet the large flow condition.

If each of a plurality of hash tables in the at least one hash tableincludes a matched statistical entry, a count value stored in a countvalue field of a matched statistical entry in one hash table may beincreased by 1, and then count values of all matched statistical entriesare added. A result obtained after addition is compared with a presetthreshold. If the result obtained after addition is greater than thepreset threshold, it may be determined that the flow to which the packetbelongs meets the large flow condition. If the result obtained afteraddition is not greater than the preset threshold, it may be determinedthat the flow to which the packet belongs does not meet the large flowcondition.

For example, a hash table 1, a hash table 2, and a hash table 3 in theat least one hash table each include a matched statistical entrymatching a packet, and count values in the hash table 1, the hash table2, and the hash table 3 are respectively 16, 29, and 31. After thepacket is received, the count value in the hash table 1 may be increasedby 1. In this case, the count value in the hash table 1 becomes 17.Then, the count values in the hash table 2 and the hash table 3 and anupdated count value in hash table 1 are added (that is, 29+31+17) andcompared with the preset threshold.

When it is determined that a flow to which a packet belongs meets thelarge flow condition, a value of a large flow flag field in astatistical entry matching the packet may be changed to 1, to indicatethat the flow to which the packet belongs is a large flow. If a flow towhich a packet belongs does not meet the large flow condition, a valueof a large flow flag field does not need to be changed, that is, thevalue of the large flow flag field remains as a default value 0, toindicate that the flow to which the packet belongs is a small flow.

In some embodiments, if no statistical entry matching the packet existsin the plurality of statistical entries, one statistical entry may becreated. An identifier in an identifier field in the created statisticalentry is an identifier of the packet. A count value in a count valuefield is 1. A value of a large flow flag field is 0, indicating that theflow to which the packet belongs is a small flow.

In some other embodiments, if no statistical entry matching the packetexists in the plurality of statistical entries, one statistical entrymay be determined from the plurality of statistical entries. Anidentifier in an identifier field in the statistical entry is changed tothe identifier of the packet. A value of a count value field is changedto 1. A value of a large flow flag field is set to 0, to indicate thatthe flow to which the packet belongs is a small flow. The determinedstatistical entry may be a statistical entry that does not match apacket for a long time.

In some embodiments, when it is determined that a flow to which a packetbelongs meets the large flow condition, a value of a large flow flagfield in a statistical entry matching the packet may be changed to 0, toindicate that the flow to which the packet belongs is a large flow. If aflow to which a packet belongs does not meet the large flow condition, avalue of a large flow flag field does not need to be changed, that is,the value of the large flow flag field remains as a default value 1, toindicate that the flow to which the packet belongs is a small flow.

In some other embodiments, the statistical entry may include only anidentifier field and a count value field. In this case, each time apacket is received, a count value in a matched statistical entry isincreased by 1, and an updated count value (or a sum of count values ofall matched statistical entries) is compared with the preset threshold.Whether a flow to which the packet belongs is a large flow is determinedbased on a comparison result.

In some other embodiments, the determining module 130 may use acount-mini sketch (CM-Sketch) to determine whether a flow to which apacket belongs meets the large flow condition.

A basic idea of using the CM-Sketch to determine whether a flow to whicha packet belongs meets the large flow condition is as follows: There isan array with a length of n for counting, and an initial value of eachelement in the array is 0. The identifier of the packet determined bythe identifier calculation module 110 is a number ranging from 0 to n.Therefore, the identifier of the packet may be used as an index of theelement. The determining module 130 obtains the identifier of the packetdetermined by the identifier calculation module 110, and may find aplurality of elements in the array based on the identifier of thepacket. The plurality of elements are in a one-to-one correspondencewith a plurality of values, and a minimum value in the plurality valuesis a count value of the flow. The minimum value of the plurality ofvalues is added by 1, and an updated value of an element (namely, avalue of the element added by 1) is compared with a preset threshold. Ifthe updated value of the element is greater than the preset threshold,it may be determined that the flow to which the packet belongs meets thelarge flow condition. If the value of the updated element is not greaterthan the preset threshold, it may be determined that the flow to whichthe packet belongs does not meet the large flow condition. If there isno element corresponding to the identifier of the packet in the array,it may be determined that the flow to which the packet belongs does notmeet the large flow condition.

Optionally, in some embodiments, a timer may be set. When the timerexpires, the sliding window count bucket or CM-Sketch-based array may bepartially or completely cleared or aged.

Division into the units shown in FIG. 1 is logical function division,and there may also be another division manner. For example, large flowcondition determining and flowlet query may be implemented by onefunction module. For example, a table used to determine whether thelarge flow condition is met may be combined with the flowlet routingtable. For ease of description, a combined table may also be referred toas a flowlet routing table. The flowlet routing table may include anidentifier field, a valid information field, and a next-hop port field.For specific functions of the identifier field, the valid informationfield, and the next-hop port field, refer to the foregoing content. Theflowlet routing table may further include a count value field and alarge flow flag field. The flowlet routing table may store some entriesthat do not meet the large flow condition. Values of large flow flagfields of these entries are negative values (for example, 0). In otherwords, values of a valid bit field and a large flow flag field of anentry that meets the large flow condition are 1. In this embodiment, anentry in the flowlet routing table may be replaced by an entrycorresponding to a newly arrived packet. For example, the flowletrouting table is constructed by using a hash table, and a location ofeach entry in the flowlet routing table is obtained through hashcalculation. When a location of the newly arrived packet is the same asthat of an existing entry, if a valid bit of the entry is 0 or it isdetermined that a flow has aged, an entry generated by the newly arrivedpacket is used to replace the existing entry. In the foregoing example,only some entries that meet the large flow condition and created entriesof some newly arrived flows (or flows that have been aged and deletedfrom the flowlet routing table) are stored. A large flow flag of thecreated entry may be 0, and a value of a large flow flag field of theentry is changed to 1 only when a count value used to determine whetherthe large flow condition is met exceeds a threshold.

(4) The Flowlet Routing Assignment Module 140

The flowlet routing assignment module 140 assigns a next-hop port for apacket.

In some embodiments, the flowlet routing assignment module 140 maydetermine the next-hop port for the packet based on a dynamic routingtable. The dynamic routing table includes a plurality of dynamicentries. Each of the plurality of dynamic entries may include anidentifier field and a next-hop port field. The identifier field mayinclude an identifier, and the next-hop port field may include aplurality of next-hop ports. In some embodiments, the dynamic routingtable may be stored on the forwarding node 100 or on another device(such as a storage device) communicating with the forwarding node.

The flowlet routing assignment module 140 may determine whether adynamic entry matching a packet to which a next-hop port is to beassigned (for ease of description, “a packet to which a next-hop port isto be assigned” is referred to as a “to-be-assigned packet” for shorthereinafter) exists in the plurality of dynamic entries. An identifierincluded in an identifier field in the dynamic entry matching theto-be-assigned packet is the same as an identifier of the to-be-assignedpacket. If the dynamic entry (which may be referred to as a matcheddynamic entry) matching the to-be-assigned packet exists, one of aplurality of next-hop ports included in a next-hop port field in thematched dynamic entry may be determined as a next-hop port assigned tothe to-be-assigned packet. For ease of description, the next-hop portassigned for the to-be-assigned packet may be referred to as a targetnext-hop port. If no dynamic entry matching the to-be-assigned packetexists, the packet may be forwarded through a pre-configured defaultnext-hop port.

Optionally, in some embodiments, the target next-hop port may be any oneof the plurality of next-hop ports.

Optionally, in some other embodiments, the flowlet routing assignmentmodule 140 may determine path quality of a transmission pathcorresponding to each of the plurality of next-hop ports, and select thetarget next-hop port based on the obtained path quality. For example, anext-hop port corresponding to a piece of path quality that meets apreset condition in the plurality of pieces of path quality may bedetermined as the target next-hop port.

The path quality may be reflected by one or more of a throughput, anend-to-end delay, a packet loss rate, or a jitter rate of a path. Forexample, the path quality may be a value of one or more of thethroughput, the delay, the packet loss rate, or the jitter rate, or apath quality score obtained through calculation based on the value ofthe one or more of the throughput, the delay, the packet loss rate, orthe jitter rate.

The preset condition may be set as required. For example, a next-hopport corresponding to a path with best path quality may be selected asthe target next-hop port, for example, a next-hop port corresponding toa path with a minimum delay, a maximum throughput, a lowest packet lossrate, or a lowest jitter rate. For another example, a next-hop portcorresponding to a path with a minimum delay may be selected, as thetarget next-hop port, from a plurality of paths with a same throughput.For another example, a next-hop port corresponding to a path with amaximum throughput in a plurality of paths whose delays meet a presetdelay threshold may be selected as the target next-hop port.

Optionally, in some other embodiments, the flowlet routing assignmentmodule 140 may select, as the target next-hop port, one of a pluralityof next-hop ports in a plurality of flowlet entries included in theflowlet routing table stored by the flowlet query module 120. Forexample, a next-hop port with a least load may be selected as the targetnext-hop port.

(5) The Flowlet Routing Forward Module 150

The flowlet routing forward module 150 may forward a packet based on anext-hop port in a flowlet entry determined by the flowlet query module120.

(6) The ECMP Routing Forward Module 160

The ECMP routing forward module 160 may store an ECMP routing table. TheECMP routing table may include a plurality of ECMP entries, and each ofthe plurality of ECMP entries may include an identifier field and anext-hop port field.

If a packet determined by the determining module 130 does not meet thelarge flow condition, the packet may be sent to the ECMP routing forwardmodule 160. The ECMP routing forward module 160 may determine whetherthere is an ECMP entry matching the packet in the plurality of ECMPentries included in the ECMP routing table. If there is an ECMP entrymatching the packet (hereinafter referred to as a matched ECMP entry),the packet is forwarded based on a next-hop port in a next-hop portfield in the matched ECMP entry. If there is no ECMP entry matching thepacket, the packet may be forwarded through a default next-hop port.

The ECMP routing table may be a static ECMP routing table. The staticECMP routing table may include a group table and a member table. A baseaddress may be queried from the group table based on flow identityinformation of the packet. A next-hop port is determined, based on thequeried base address and an offset value, from the member table toforward the packet. The offset value is determined based on the flowidentity information of the packet. For example, the offset value may bedetermined based on a hash value of the flow identity information, ormay be determined based on an identifier of the packet determined by theidentifier calculation module 110. One base address corresponds to aplurality of offset values, and each of the plurality of offset valuescorresponds to one next-hop port.

The ECMP routing table may be a static ECMP routing table. The staticECMP routing table may include a group table and a member table. A baseaddress may be queried from the group table based on flow identityinformation of the packet. A next-hop port is determined, based on thequeried base address and an offset value, from the member table toforward the packet. The offset value is determined based on the flowidentity information of the packet. For example, the offset value may bedetermined based on a hash value of the flow identity information, ormay be determined based on an identifier of the packet determined by theidentifier calculation module 110. One base address corresponds to aplurality of offset values, and each of the plurality of offset valuescorresponds to one next-hop port.

The ECMP routing table may be a static ECMP routing table. The staticECMP routing table may include a group table and a member table. A baseaddress may be queried from the group table based on flow identityinformation of the packet. A next-hop port is determined, based on thequeried base address and an offset value, from the member table toforward the packet. The offset value is determined based on the flowidentity information of the packet. For example, the offset value may bedetermined based on a hash value of the flow identity information, ormay be determined based on an identifier of the packet determined by theidentifier calculation module 110. One base address corresponds to aplurality of offset values, and each of the plurality of offset valuescorresponds to one next-hop port.

After determining an identifier of a received packet, the forwardingnode 100 may process the packet in two manners.

Processing manner 1: The determining module 130 first determines, basedon the identifier of the packet, whether a flow to which the packetbelongs meets the large flow condition. If the large flow condition ismet, the flowlet query module 120 determines whether there is a matchedflowlet entry, and determines, based on a determined result, to forwardthe packet based on the matched flowlet entry, a created flowlet entry,or an updated flowlet entry. If the large flow condition is not met, theECMP routing forward module 160 forwards the packet based on a matchedECMP entry.

Processing manner 2: The flowlet query module 120 determines, based onthe identifier of the packet, whether there is a matched flowlet entryin the flowlet routing table. If there is a matched flowlet entry, thepacket is forwarded based on the matched flowlet entry or an updatedflowlet entry. If there is no matched flowlet entry, the determiningmodule 130 determines, based on the identifier of the packet, whether aflow to which the packet belongs meets the large flow condition. If thelarge flow condition is met, a flowlet entry is created, and the packetis forwarded based on the created flowlet entry. If the large flowcondition is not met, the ECMP routing forward module 160 forwards thepacket based on a matched ECMP entry.

FIG. 2 is a flowchart of packet processing corresponding to theprocessing manner 1.

201: Determine, based on flow identity information of a packet, anidentifier of the packet.

202: Determine, based on the identifier of the packet, whether a flow towhich the packet belongs meets a large flow condition.

If the flow to which the packet belongs does not meet the large flowcondition, operation 203 is performed. If the flow to which the packetbelongs meets the large flow condition, operation 204 is performed.

203: Forward the packet based on an ECMP routing table.

204: Determine whether there is a flowlet entry matching the identifierof the packet in a flowlet routing table.

If there is a flowlet entry matching the identifier of the packet in theflowlet routing table, operation 205 is performed. If there is noflowlet entry matching the identifier of the packet in the flowletrouting table, operation 209 and operation 210 are performed.

205: Determine whether the matched flowlet entry needs to be updated.

If the matched flowlet entry does not need to be updated, step 206 maybe performed. If the matched flowlet entry needs to be updated,operation 207 and operation 208 may be performed.

Whether the matched flowlet entry needs to be updated may be determinedbased on a valid information field (to be specific, a valid bit fieldand an aging flag field, or a valid bit field and a timestamp field) inthe flowlet entry matching the identifier of the packet. As describedabove, when the valid bit field is 1, whether a flowlet to which thepacket belongs is aged may be determined based on the aging flag fieldor the timestamp field. If a determined result is that the flowlet isaged, the valid bit field needs to be set to 0. In this case, it may bedetermined that the matched flowlet entry needs to be updated. If thedetermined result is that the flowlet is not aged, a value of the validbit field remains unchanged. In this case, it may be determined that thematched flowlet entry does not need to be updated.

206: Forward the packet based on the matched flowlet entry.

207: Update the matched flowlet entry.

208: Forward the packet based on an updated matched flowlet entry.

209: Create a flowlet entry in the flowlet routing table.

210: Forward the packet based on the created flowlet entry.

FIG. 3 is a schematic diagram of a relationship between the modules inthe forwarding node 100 when the packet is processed in the processingmanner 1.

As shown in FIG. 3 , the identifier calculation module 110 may determinethe identifier of the packet, and send the determined identifier of thepacket to the determining module 130.

The determining module 130 determines whether the flow to which thepacket belongs meets the large flow condition. If the large flowcondition is met, the flowlet query module 120 processes the packet. Ifthe large flow condition is not met, the ECMP routing forward module 160forwards the packet based on the stored ECMP routing table.

The flowlet query module 120 determines whether there is a flowlet entrymatching the packet in the stored flowlet routing table.

(1) If there is a matched flowlet entry and the matched flowlet entrydoes not need to be updated, the flowlet routing forward module 150forwards the packet.

(2) If there is no matched flowlet entry, the flowlet routing assignmentmodule 140 assigns a new next-hop port and creates a flowlet entry basedon the assigned new next-hop port. Then, the flowlet routing forwardmodule 150 forwards the packet based on the created flowlet entry.

(3) If there is a matched flowlet entry and the matched flowlet entryneeds to be updated, the flowlet routing assignment module 140 assigns anew next-hop port and updates the matched flowlet entry based on theassigned new next-hop port. Then, the flowlet routing forward module 150forwards the packet based on an updated matched flowlet entry.

FIG. 4 is a flowchart of packet processing corresponding to theprocessing manner 2.

401: Determine, based on flow identity information of a packet, anidentifier of the packet.

402: Determine whether there is a flowlet entry matching the identifierof the packet in a flowlet routing table.

If there is a flowlet entry matching the identifier of the packet in theflowlet routing table, operation 403 is performed. If there is noflowlet entry matching the identifier of the packet in the flowletrouting table, operation 407 is performed.

403: Determine whether the matched flowlet entry needs to be updated.

If the matched flowlet entry does not need to be updated, operation 404may be performed. If the matched flowlet entry needs to be updated,operation 405 and operation 406 are performed.

404: Forward the packet based on the matched flowlet entry.

405: Update the matched flowlet entry.

406: Forward the packet based on an updated matched flowlet entry.

407: Determine whether a flow to which the packet belongs meets a largeflow condition.

If the flow to which the packet belongs meets the large flow condition,operation 408 and operation 409 are performed. If the flow to which thepacket belongs does not meet the large flow condition, operation 410 isperformed.

408: Create a flowlet entry in the flowlet routing table.

409: Forward the packet based on the created flowlet entry.

410: Forward the packet based on an ECMP routing table.

FIG. 5 is a schematic diagram of a relationship between the modules inthe forwarding node 100 when the packet is processed in the processingmanner 2.

As shown in FIG. 5 , the identifier calculation module 110 may determinethe identifier of the packet and send the identifier of the packet tothe flowlet query module 120.

The flowlet query module 120 determines whether there is a flowlet entrymatching the packet in the stored flowlet routing table.

(1) If there is a matched flowlet entry and the matched flowlet entrydoes not need to be updated, the flowlet routing forward module 150forwards the packet.

(2) If there is a matched flowlet entry and the matched flowlet entryneeds to be updated, the flowlet routing assignment module 140 assigns anew next-hop port and updates the matched flowlet entry based on theassigned new next-hop port. Then, the flowlet routing forward module 150forwards the packet based on an updated matched flowlet entry.

(3) If there is no matched flowlet entry, the determining module 130continues processing.

The determining module 130 determines whether the flow to which thepacket belongs meets the large flow condition. If the large flowcondition is not met, the ECMP routing forward module 160 forwards thepacket based on the stored ECMP routing table. If the large flowcondition is met, the flowlet routing assignment module 140 assigns anew next-hop port and creates a new flowlet entry based on the assignednew next-hop port. Then, the flowlet routing forward module 150 forwardsthe packet based on the created flowlet entry.

To help a person skilled in the art understand the technical solutionsin this application better, the following describes the technicalsolutions in this application with reference to specific embodiments.

For ease of description, a flow A, a flow B, and a flow C are used torepresent three different flows received by the forwarding node 100. Inother words, flow identity information of any two of the flow A, theflow B, and the flow C is different. A packet A₁ is used to represent afirst packet in the flow A received by the forwarding node 100, a packetA₂ is used to represent a second packet in the flow A received by theforwarding node 100, a packet A₃ is used to represent a third packet inthe flow A received by the forwarding node 100, and so on. Similarly, apacket B₁ is used to represent a first packet in the flow B received bythe forwarding node 100, a packet B₂ is used to represent a secondpacket in the flow B received by the forwarding node 100, a packet B₃ isused to represent a third packet in the flow B received by theforwarding node 100, and so on. Similarly, a packet C₁ is used torepresent a first packet in the flow C received by the forwarding node100, a packet C₂ is used to represent a second packet in the flow Creceived by the forwarding node 100, a packet C₃ is used to represent athird packet in the flow C received by the forwarding node 100, and soon.

The packet A₁, the packet A₂, the packet A₃, . . . , and a packet A_(X)are all packets in the flow A (where X is a positive integer greaterthan 3). Therefore, flow identity information of any two packets fromthe packet A₁ to the packet A_(X) is the same, and ID_(A) may be used torepresent an identifier determined based on the flow identityinformation of the packets in the flow A.

Similarly, the packet B₁, the packet B₂, the packet B₃, . . . , and apacket By are all packets in the flow B (where Y is a positive integergreater than 3). Therefore, flow identity information of any two packetsfrom the packet B₁ to the packet By is the same, and ID_(B) may be usedto represent an identifier determined based on the flow identityinformation of the packets in the flow B.

Similarly, the packet C₁, the packet C₂, the packet C₃, . . . , and apacket C_(Z) are all packets in the flow C (where Z is a positiveinteger greater than 3). Therefore, flow identity information of any twopackets from the packet C₁ to the packet C_(Z) is the same, and ID_(C)may be used to represent an identifier determined based on the flowidentity information of the packets in the flow C.

It is assumed that the forwarding node 100 is in an initial state whenaccessing a network. In other words, the forwarding node 100 has notreceived any packet before. In this case, the flowlet routing tablestored by the flowlet query module 120 of the forwarding node 100 is anempty routing table. For ease of description, in the following, it isassumed that fields included in a flowlet entry in the flowlet routingtable are: an identifier field, a valid bit field, an aging flag field,and a next-hop port field. A hash table in a sliding window countingbucket or an array in a CM-Sketch that is used by the determining module130 to determine whether a flow meets the large flow condition is alsoempty. For ease of description, in the following, it is assumed that thedetermining module 130 determines, based on the sliding window countingbucket, whether the flow to which the packet belongs meets the largeflow condition and that a flow whose count value is greater than orequal to 100 is considered as a large flow. The ECMP routing tablestored in the ECMP routing forward module 160 may be an ECMP routingtable in which a next-hop port field is preset, and the ECMP routingtable includes a group table and a member table. In other words, in thiscase, the flowlet routing table, the sliding window counting bucket, andthe ECMP routing table are respectively shown in Table 1, Table 2, andTable 3. For ease of description, Table 3 shows only the member table inthe ECMP routing table.

TABLE 1 Identifier Valid bit Aging flag Next-hop port  

TABLE 2 Identifier Count value Large flow flag  

TABLE 3 Base address Offset value Next-hop port Base₀ Offset_(A) Port 1Offset_(B) Port 2 Offset_(C) Port 3

As described above, Table 1 (namely, the flowlet routing table) is anempty routing table, Table 2 (namely, the sliding window countingbucket) is also an empty table, and Table 3 (namely, the member table inthe ECMP routing table) is a routing table in which the next-hop portsare configured.

It is assumed that the forwarding node 100 receives the packet A₁, andthe packet A₁ is a first packet received by the forwarding node 100.

It is assumed that the forwarding node 100 processes the packetaccording to a procedure shown in FIG. 2 . In this case, the forwardingnode 100 may determine, based on the sliding window counting bucketshown in Table 2, whether the flow to which the packet A₁ belongs meetsthe large flow condition. Because there is no data in the sliding windowcounting bucket, related information of the packet A₁ may be added tothe sliding window counting bucket, to obtain a sliding window countingbucket shown in Table 4.

TABLE 4 Identifier Count value Large flow flag ID_(A) 1 0

As shown in Table 4, the forwarding node 100 receives only one packetwhose identifier is ID_(A), and therefore, a count value is 1. Asdescribed above, a flow whose count value is greater than or equal to100 is considered a large flow. Because currently the count value isonly 1, a large flow flag is 0. In other words, the flow to which thepacket A₁ belongs is not a large flow.

In this case, the forwarding node 100 may forward the packet A₁ based onthe ECMP routing table. It is assumed that the flow identity informationof the packet in the flow A is Base₀. It is assumed that an offset valuedetermined based on the flow identity information of the packet in theflow A is Offset_(A). In this case, it may be determined, based on Table3, that a next-hop port for the packet A₁ is a port 1. It is assumedthat the forwarding node 100 processes the packet according to aprocedure shown in FIG. 4 . In this case, the forwarding node 100 needsto first determine whether there is a flowlet entry whose identifier inan identifier field is ID_(A) in a current flowlet routing table,namely, a flowlet entry matching the packet A₁. As shown in Table 1, thecurrent flowlet routing table is an empty routing table. Therefore,there is no flowlet entry matching the packet A₁. In this case, theforwarding node 100 may determine whether the flow to which the packetA₁ belongs meets the large flow condition. A specific process in whichthe forwarding node 100 may determine whether the flow to which thepacket A₁ belongs meets the large flow condition and a subsequentprocedure are the same as corresponding operations in which theforwarding node processes the packet according to the procedure shown inFIG. 2 . For brevity, details are not described herein again.

It is assumed that the forwarding node 100 receives the packet B₁, andthe packet B₁ is a second packet received by the forwarding node 100.

It is assumed that the forwarding node 100 processes the packetaccording to the procedure shown in FIG. 2 . In this case, theforwarding node 100 may determine, based on the sliding window countingbucket shown in Table 2, whether the flow to which the packet B₁ belongsmeets the large flow condition. Because the sliding window countingbucket does not include an entry matching the flow to which the packetB₁ belongs, the entry matching the flow to which the packet B₁ belongsmay be created, to obtain a sliding window counting bucket shown inTable 5.

TABLE 5 Identifier Count value Large flow flag ID_(A) 1 0 ID_(B) 1 0

As shown in Table 5, the forwarding node 100 receives only one packetwhose identifier is ID_(B), and therefore, a count value is 1. Asdescribed above, a flow whose count value is greater than or equal to100 is considered a large flow. Because currently the count value isonly 1, a large flow flag is 0. In other words, the flow to which thepacket B₁ belongs is not a large flow.

In this case, the forwarding node 100 may forward the packet B₁ based onthe ECMP routing table. It is assumed that a base address determinedbased on the flow identity information of the packet in the flow B isalso Base₀. It is assumed that an offset value determined based on theflow identity information of the packet in the flow B is Offset_(B). Inthis case, it may be determined, based on Table 3, that a next-hop portfor the packet B₁ is the port 2.

It is assumed that the forwarding node 100 processes the packetaccording to the procedure shown in FIG. 4 . In this case, theforwarding node 100 needs to first determine whether there is a flowletentry whose identifier in an identifier field is ID_(B) in the currentflowlet routing table, namely, a flowlet entry matching the packet B₁.As shown in Table 1, the current flowlet routing table is an emptyrouting table. Therefore, there is no flowlet entry matching the packetB₁. In this case, the forwarding node 100 may determine whether the flowto which the packet B₁ belongs meets the large flow condition. Aspecific process in which the forwarding node 100 may determine whetherthe flow to which the packet B₁ belongs meets the large flow conditionand a subsequent procedure are the same as corresponding operations inwhich the forwarding node processes the packet according to theprocedure shown in FIG. 2 . For brevity, details are not describedherein again.

It is assumed that the forwarding node 100 receives the packet C₁, andthe packet C₁ is a third packet received by the forwarding node 100. Amanner in which the forwarding node 100 processes the packet C₁ issimilar to manners in which the forwarding node 100 processes the packetA₁ and the packet B₁. For brevity, details are not described hereinagain. After the packet C₁ is processed, the sliding window countingbucket may be updated, as shown in Table 6.

TABLE 6 Identifier Count value Large flow flag ID_(A) 1 0 ID_(B) 1 0ID_(C) 1 0

It is assumed that the forwarding node 100 successively receives thepacket A₂ to a packet A₉₉, the packet B₂ to a packet B₈₆, and the packetC₂ to a packet C₁₆. In this case, the sliding window counting bucket isupdated, as shown in Table 7.

TABLE 7 Identifier Count value Large flow flag ID_(A) 99 0 ID_(B) 86 0ID_(C) 16 0

It can be learned that, so far, count values corresponding to the flowA, the flow B, and the flow C are not greater than or equal to thepreset threshold 100. Therefore, the flow A, the flow B, and the flow Call do not meet the large flow condition, and corresponding large flowflags are 0. The packets A₂ to A₉₉, the packets B₂ to B₈₆, and thepacket C₂ to C₁₆ are all forwarded based on the ECMP routing table shownin Table 3.

It is assumed that the forwarding node 100 receives a packet A₁₀₀, andthe forwarding node 100 processes the packet A₁₀₀ according to theprocedure shown in FIG. 2 . In this case, the forwarding node 100updates Table 7, as shown in Table 8.

TABLE 8 Identifier Count value Large flow flag ID_(A) 100 1 ID_(B) 86 0ID_(C) 16 0

As shown in Table 8, after the packet A₁₀₀ is received, a count value ofan entry (namely, an entry whose identifier field is ID_(A))corresponding to the flow to which the packet A₁₀₀ belongs is equal tothe preset threshold. Therefore, the large flow flag may be set to 1. Inthis case, the flow A is considered to meet the large flow condition.

The forwarding node 100 determines whether the flowlet routing tableincludes a flowlet entry matching the packet A₁₀₀ (namely, a flowletentry whose identifier in an identifier field is ID_(A)). Because theflow A is a first flow that is determined to meet the large flowcondition, no flowlet entry is created before. Therefore, so far, theflowlet routing table is still an empty routing table. In this case, theflowlet routing table does not include the flowlet entry matching thepacket A₁₀₀. Therefore, the flowlet entry matching the packet A₁₀₀ maybe created in the flowlet routing table. Because the packet A₁₀₀ is apacket in the flow A, the flowlet entry matching the packet A₁₀₀ mayalternatively be considered as a flowlet entry matching the flow A. Itis assumed that a determined next-hop port used to forward the flow A isa port 11. In this case, the flowlet routing table shown in Table 1 maybe updated, as shown in Table 9. A value of an aging flag field in Table9 is a value of the aging flag field before the timer expires.

TABLE 9 Identifier Valid bit Aging flag Next-hop port ID_(A) 1 1 Port 11

In this case, the forwarding node 100 may forward the packet A₁₀₀ basedon the created flowlet entry.

It is assumed that the forwarding node 100 receives a packet A₁₀₀, andit is assumed that the forwarding node 100 processes the packet A₁₀₀according to the procedure shown in FIG. 4 . In this case, theforwarding node 100 determines whether the flowlet routing tableincludes a flowlet entry matching the packet A₁₀₀ (namely, a flowletentry whose identifier in an identifier field is ID_(A)). So far, noflow A has been determined as a flow that meets the large flowcondition. Therefore, so far, the flowlet routing table is still anempty routing table. In this case, the flowlet routing table does notinclude the flowlet entry matching the packet A₁₀₀.

The forwarding node 100 continues to determine whether the flow A meetsthe large flow condition. After receiving the packet A₁₀₀, theforwarding node 100 updates Table 7, as shown in Table 8. As shown inTable 8, after the packet A₁₀₀ is received, a count value of an entry(namely, an entry whose identifier field is ID_(A)) corresponding to theflow to which the packet A₁₀₀ belongs is equal to the preset threshold.Therefore, the large flow flag may be set to 1. In this case, the flow Ais considered to meet the large flow condition. In this case, theforwarding node 100 may create the flowlet entry matching the packetA₁₀₀ in the flowlet routing table. It is assumed that a determinednext-hop port used to forward the flow A is a port 11. In this case, theflowlet routing table shown in Table 1 may be updated, as shown in Table9. The forwarding node 100 may forward the packet A₁₀₀ based on thecreated flowlet entry.

It is assumed that the forwarding node 100 also receives a packet A₁₀₁,and the packet A₁₀₁ does not time out (in other words, the packet A₁₀₁and the packet A₁₀₀ belong to a same flowlet).

If the forwarding node 100 processes the packet A₁₀₁ according to theprocedure shown in FIG. 2 , the forwarding node 100 first updates anentry corresponding to the flow A in the sliding window counting bucket,to obtain a table shown in Table 10.

TABLE 10 Identifier Count value Large flow flag ID_(A) 101 1 ID_(B) 86 0ID_(C) 16 0

As shown in Table 10, a large flow flag of the entry corresponding tothe flow A is 1. In this case, the flow A is considered to meet thelarge flow condition.

The forwarding node 100 determines whether the flowlet routing tableincludes a flowlet entry matching the packet A₁₀₁ (namely, a flowletentry whose identifier in an identifier field is ID_(A)). It is assumedthat the packet A₁₀₁ does not time out, a flowlet routing table used todetermine whether the flowlet entry matching the packet A₁₀₁ is includedis a routing table shown in Table 9. As shown in Table 9, the flowletrouting table includes the flowlet entry whose identifier in theidentifier field is ID_(A), and both a valid bit and an aging flagare 1. In this case, the forwarding node 100 may forward the packet A₁₀₁based on the flowlet entry matching the packet A₁₀₁.

If the forwarding node 100 processes the packet A₁₀₁ according to theprocedure shown in FIG. 4 , the forwarding node 100 determines whetherthe flowlet routing table includes a flowlet entry matching the packetA₁₀₁ (namely, a flowlet entry whose identifier in an identifier field isID_(A)). It is assumed that the packet A₁₀₁ does not time out, a flowletrouting table used to determine whether the flowlet entry matching thepacket A₁₀₀ is included is a routing table shown in Table 9. As shown inTable 9, the flowlet routing table includes the flowlet entry whoseidentifier in the identifier field is ID_(A), and both a valid bit andan aging flag are 1. In this case, the forwarding node 100 may determinethat the packet A₁₀₁ is not a first packet of a new flowlet. Therefore,the forwarding node 100 may forward the packet A₁₀₁ based on the flowletentry matching the packet A₁₀₁.

It is assumed that the forwarding node 100 successively receives apacket A₁₀₂ to a packet A₁₂₀, a packet B₈₇ to a packet B₁₂₂, and apacket C₁₇ to a packet C₈₆. In this case, the sliding window countbucket is updated, as shown in Table 11.

TABLE 11 Identifier Count value Large flow flag ID_(A) 120 1 ID_(B) 1221 ID_(C)  86 0

The flowlet routing table is updated, as shown in Table 12. A value ofan aging flag field in Table 12 is a value of the aging flag fieldbefore the timer expires.

TABLE 12 Identifier Valid bit Aging flag Next-hop port ID_(A) 1 1 Port11 ID_(B) 1 0 Port 12

It is assumed that the forwarding node 100 receives a packet B₁₂₃ andthe packet B₁₂₃ times out. It is assumed that the forwarding node 100processes the packet B₁₂₃ according to the procedure shown in FIG. 2 .In this case, the forwarding node 100 may determine that the flow B towhich the packet B₁₂₃ belongs meets the large flow condition. Theforwarding node 100 determines whether there is a matched flowlet entryin the flowlet routing table shown in Table 12. There is a matchedflowlet entry in the flowlet routing table shown in Table 12. However,as shown in Table 12, the value of the aging flag field is 0. Therefore,a value of the valid bit in Table 12 needs to be set to 0. In this case,the matched flowlet entry needs to be updated, and a new next-hop portneeds to be reassigned for the flow B. It is assumed that the newnext-hop port reassigned for flow B is a port 18. In this case, aflowlet entry matching the flow B in the flowlet routing table shown inTable 12 needs to be updated, to obtain a flowlet routing table shown inTable 13. A value of an aging flag field in Table 13 is a value of theaging flag field before the timer expires.

TABLE 13 Identifier Valid bit Aging flag Next-hop port ID_(A) 1 1 Port11 ID_(B) 1 1 Port 18

In this case, the forwarding node 100 may forward the packet B₁₂₃through the port 18 based on an updated flowlet entry matching the flowB.

The forwarding node in the embodiments of this application may be anetwork device (for example, a switch or a router) having an IP packetforwarding function. The steps implemented by the forwarding node mayalternatively be implemented by a component in the network device, forexample, may be implemented by a chip system in the network device. Thechip system includes a logic circuit. The logic circuit may be coupledto an input/output interface, and transmit a packet through theinput/output interface, to implement the operations in the foregoingmethod embodiments. The logic circuit may be a field programmable gatearray (FPGA), an application-specific integrated circuit (ASIC), anetwork processor (NP), another programmable logic device, a discretegate or a transistor logic device, a discrete hardware component,another integrated chip, or the like.

In an embodiment, operations in the foregoing method can be implementedby using a hardware integrated logical circuit in a processor, or byusing instructions in a form of software. The operations of the methoddisclosed with reference to the embodiments of this application may bedirectly performed by a hardware processor, or may be performed by usinga combination of hardware in the processor and a software module. Thesoftware module may be located in a mature storage medium in the art,such as a random access memory, a flash memory, a read-only memory, aprogrammable read-only memory, an electrically erasable programmablememory, or a register. The storage medium is located in a memory, andthe processor reads information in the memory and completes theoperations in the foregoing method in combination with hardware of theprocessor. To avoid repetition, details are not described herein again.

It may be understood that the memory in the embodiments of thisapplication may be a volatile memory or a nonvolatile memory, or mayinclude a volatile memory and a nonvolatile memory. The nonvolatilememory may be a read-only memory (ROM), a programmable read-only memory(PROM), an erasable programmable read-only memory (EPROM), anelectrically erasable programmable read-only memory (EEPROM), or a flashmemory. The volatile memory may be a random access memory (RAM), and isused as an external cache. Through example but not limitativedescription, many forms of RAMs may be used, for example, a staticrandom access memory (SRAM), a dynamic random access memory (DRAM), asynchronous dynamic random access memory (SDRAM), a double data ratesynchronous dynamic random access memory (DDR SDRAM), an enhancedsynchronous dynamic random access memory (ESDRAM), a synchlink dynamicrandom access memory (SLDRAM), and a direct rambus random access memory(DR RAM). It should be noted that the memory in the system and methoddescribed in this specification includes but is not limited to these andany memory of another proper type.

According to the method provided in the embodiments of this application,this application further provides a computer program product. Thecomputer program product includes computer program code. When thecomputer program code is run on a network device, the network device isenabled to perform the method in any one of the foregoing embodiments.

According to the method provided in the embodiments of this application,this application further provides a computer-readable medium. Thecomputer-readable medium stores program code. When the program code isrun on a network device, the network device is enabled to perform themethod in any one of the foregoing embodiments.

A person of ordinary skill in the art may be aware that, in combinationwith the examples described in the embodiments disclosed in thisspecification, units and algorithm operations may be implemented byelectronic hardware or a combination of computer software and electronichardware. Whether the functions are performed by hardware or softwaredepends on particular applications and design constraint conditions ofthe technical solutions. A person skilled in the art may use differentmethods to implement the described functions for each particularapplication, but it should not be considered that the implementationgoes beyond the scope of this application.

It may be clearly understood by a person skilled in the art that, forthe purpose of convenient and brief description, for a detailed workingprocess of the foregoing system, apparatus, and unit, refer to acorresponding process in the foregoing method embodiments, and detailsare not described herein again.

In the several embodiments provided in this application, it should beunderstood that the disclosed system, apparatus, and method may beimplemented in other manners. For example, the described apparatusembodiment is merely an example. For example, the unit division ismerely logical function division and may be other division in an actualimplementation. For example, a plurality of units or components may becombined or integrated into another system, or some features may beignored or not performed. In addition, the displayed or discussed mutualcouplings or direct couplings or communication connections may beimplemented through some interfaces. The indirect couplings orcommunication connections between the apparatuses or units may beimplemented in electronic, mechanical, or other forms.

The units described as separate parts may or may not be physicallyseparate, and parts displayed as units may or may not be physical units,may be located in one position, or may be distributed on a plurality ofnetwork units. Some or all of the units may be selected based on actualrequirements to achieve the objectives of the solutions of theembodiments.

In addition, functional units in the embodiments of this application maybe integrated into one processing unit, or each of the units may existalone physically, or two or more units are integrated into one unit.

When the functions are implemented in a form of a software functionalunit and sold or used as an independent product, the functions may bestored in a computer-readable storage medium. Based on such anunderstanding, the technical solutions of this application essentially,or the part contributing to the conventional technology, or some of thetechnical solutions may be implemented in a form of a software product.The computer software product is stored in a storage medium, andincludes several instructions for instructing a computer device (whichmay be a personal computer, a server, or a network device) to performall or some of the operations of the methods described in theembodiments of this application. The foregoing storage medium includesany medium that can store program code, such as a USB flash drive, aremovable hard disk, a read-only memory (ROM), a random access memory(RAM), a magnetic disk, or a compact disc.

The foregoing descriptions are merely specific implementations of thisapplication, but are not intended to limit the protection scope of thisapplication. Any variation or replacement readily figured out by aperson skilled in the art within the technical scope disclosed in thisapplication shall fall within the protection scope of this application.Therefore, the protection scope of this application shall be subject tothe protection scope of the claims.

What is claimed is:
 1. A packet forwarding method, comprising:determining a first identifier of a first packet based on flow identityinformation of the first packet, wherein the first packet belongs to afirst flow; and when the first flow is a large flow and a flowletrouting table does not comprise a first flowlet entry corresponding tothe first flow, creating the first flowlet entry in the flowlet routingtable, and forwarding the first packet based on the first flowlet entry,wherein the first flowlet entry comprises the first identifier; whereinthe creating the first flowlet entry comprises: determining a firstdynamic entry from a dynamic routing table based on the firstidentifier, wherein an identifier field of the first dynamic entrycomprises the first identifier; and determining a first next-hop port ina plurality of next-hop ports comprised in the first dynamic entry,wherein the first next-hop port is a next-hop port comprised in anext-hop port field in the first flowlet entry; wherein the determiningthe first next-hop port in the plurality of next-hop ports comprised inthe first dynamic entry comprises: obtaining a plurality of pieces ofpath quality, wherein the plurality of pieces of path quality are in aone-to-one correspondence with the plurality of next-hop ports; anddetermining a next-hop port corresponding to a piece of path qualitythat meets a preset condition in the plurality of pieces of path qualityas the first next-hop port.
 2. The method according to claim 1, whereinthe flowlet routing table comprises at least one flowlet entry, and theat least one flowlet entry is in a one-to-one correspondence with atleast one flow.
 3. The method according to claim 2, wherein the at leastone flowlet entry is a valid entry in the flowlet routing table.
 4. Themethod according to claim 1, wherein the method further comprises:determining a second identifier of a second packet based on flowidentity information of the second packet, wherein the second packetincludes a second flow; and when the second flow is not a large flow,forwarding the second packet based on the second identifier and anequal-cost multi-path routing (ECMP) routing table.
 5. The methodaccording to claim 4, wherein the method further comprises: receiving athird packet; determining a third identifier of the third packet basedon flow identity information of the third packet, wherein the thirdpacket includes a third flow; determining, based on the thirdidentifier, whether the flowlet routing table comprises a third flowletentry, wherein the third flowlet entry comprises the third identifier;and if the flowlet routing table comprises the third flowlet entry,forwarding the third packet based on the third flowlet entry, or if theflowlet routing table does not comprise the third flowlet entry,determining whether the third flow is a large flow, and based on adetermined result, creating the third flowlet entry in the flowletrouting table or forwarding the third packet based on the ECMP routingtable.
 6. The method according to claim 4, wherein the method furthercomprises: receiving a fourth packet; determining a fourth identifier ofthe fourth packet based on flow identity information of the fourthpacket, wherein the fourth packet includes a fourth flow; determiningwhether the fourth flow is a large flow; and when the fourth flow is alarge flow, forwarding the fourth packet based on a fourth flowlet entryin the flowlet routing table, wherein the fourth flowlet entry comprisesthe fourth identifier, or when the fourth flow is not a large flow,forwarding the fourth packet based on the ECMP routing table.
 7. Anetwork device, comprising: a processor, wherein the processor isconfigured to execute program instructions to perform operationscomprising: determining a first identifier of a first packet based onflow identity information of the first packet, wherein the first packetbelongs to a first flow; determining whether the first flow to which thefirst packet belongs is a large flow; when the first flow is a largeflow and a flowlet routing table does not comprise a first flowlet entrycorresponding to the first flow, creating the first flowlet entry in theflowlet routing table based on a next-hop port determined by a flowletrouting assignment module, wherein the first flowlet entry comprises thefirst identifier; and forwarding the first packet based on the firstflowlet entry; wherein the creating the first flowlet entry comprises:determining a first dynamic entry from a dynamic routing table based onthe first identifier, wherein an identifier field of the first dynamicentry comprises the first identifier; and determining a first next-hopport in a plurality of next-hop ports comprised in the first dynamicentry, wherein the first next-hop port is a next-hop port comprised in anext-hop port field in the first flowlet entry; wherein the determiningthe first next-hop port in the plurality of next-hop ports comprised inthe first dynamic entry comprises: obtaining a plurality of pieces ofpath quality, wherein the plurality of pieces of path quality are in aone-to-one correspondence with the plurality of next-hop ports; anddetermining a next-hop port corresponding to a piece of path qualitythat meets a preset condition in the plurality of pieces of path qualityas the first next-hop port.
 8. The network device according to claim 7,wherein the flowlet routing table comprises at least one flowlet entry,and the at least one flowlet entry is in a one-to-one correspondencewith at least one flow.
 9. The network device according to claim 8,wherein the at least one flowlet entry is a valid entry in the flowletrouting table.
 10. The network device according to claim 7, wherein theoperations further comprise: determining a second identifier of a secondpacket based on flow identity information of the second packet, whereinthe second packet includes a second flow; determining that the secondflow is not a large flow; and when the second flow is not a large flow,forwarding the second packet based on the second identifier and anequal-cost multi-path routing (ECMP) routing table in the networkdevice.
 11. The network device according to claim 10, wherein theoperations further comprise: determining a third identifier of a thirdpacket based on flow identity information of the third packet, whereinthe third packet includes a third flow; determining, based on the thirdidentifier, whether the flowlet routing table comprises a third flowletentry, wherein the third flowlet entry comprises the third identifier;when the flowlet routing table comprises the third flowlet entry,forwarding the third packet based on the third flowlet entry; when theflowlet routing table does not comprise the third flowlet entry,determining whether the third flow is a large flow; and creating thethird flowlet entry in the flowlet routing table based on a determinedresult and the next-hop port determined by the flowlet routingassignment module, or forwarding the third packet based on a determinedresult and the ECMP routing table.
 12. The network device according toclaim 10, wherein the operations further comprise: determining a fourthidentifier of a fourth packet based on flow identity information of thefourth packet, wherein the fourth packet includes a fourth flow;determining whether the fourth flow is a large flow; and when the fourthflow is a large flow, forwarding the fourth packet based on a fourthflowlet entry in the flowlet routing table, wherein the fourth flowletentry comprises the fourth identifier, or when the fourth flow does notmeet a large flow condition, forwarding the fourth packet based on theECMP routing table.
 13. A chip system, comprising: a logic circuit,wherein the logic circuit is configured to be coupled to an input/outputinterface, and transmit data through the input/output interface toperform operations comprising: determining a first identifier of a firstpacket based on flow identity information of the first packet, whereinthe first packet belongs to a first flow; and when the first flow is alarge flow and a flowlet routing table does not comprise a first flowletentry corresponding to the first flow, creating the first flowlet entryin the flowlet routing table, and forwarding the first packet based onthe first flowlet entry, wherein the first flowlet entry comprises thefirst identifier; wherein the creating the first flowlet entrycomprises: determining a first dynamic entry from a dynamic routingtable based on the first identifier, wherein an identifier field of thefirst dynamic entry comprises the first identifier; and determining afirst next-hop port in a plurality of next-hop ports comprised in thefirst dynamic entry, wherein the first next-hop port is a next-hop portcomprised in a next-hop port field in the first flowlet entry; whereinthe determining the first next-hop port in the plurality of next-hopports comprised in the first dynamic entry comprises: obtaining aplurality of pieces of path quality, wherein the plurality of pieces ofpath quality are in a one-to-one correspondence with the plurality ofnext-hop ports; and determining a next-hop port corresponding to a pieceof path quality that meets a preset condition in the plurality of piecesof path quality as the first next-hop port.
 14. The chip systemaccording to claim 13, wherein the flowlet routing table comprises atleast one flowlet entry, and the at least one flowlet entry is in aone-to-one correspondence with at least one flow.
 15. The chip systemaccording to claim 14, wherein the at least one flowlet entry is a validentry in the flowlet routing table.
 16. The chip system according toclaim 13, wherein the operations further comprises: determining a secondidentifier of a second packet based on flow identity information of thesecond packet, wherein the second packet includes a second flow; andwhen the second flow is not a large flow, forwarding the second packetbased on the second identifier and an equal-cost multi-path routing(ECMP) routing table.
 17. The chip system according to claim 16, whereinthe operations further comprises: receiving a third packet; determininga third identifier of the third packet based on flow identityinformation of the third packet, wherein the third packet includes athird flow; determining, based on the third identifier, whether theflowlet routing table comprises a third flowlet entry, wherein the thirdflowlet entry comprises the third identifier; and if the flowlet routingtable comprises the third flowlet entry, forwarding the third packetbased on the third flowlet entry, or if the flowlet routing table doesnot comprise the third flowlet entry, determining whether the third flowis a large flow, and based on a determined result, creating the thirdflowlet entry in the flowlet routing table or forwarding the thirdpacket based on the ECMP routing table.
 18. The chip system according toclaim 16, wherein the operations further comprises: receiving a fourthpacket; determining a fourth identifier of the fourth packet based onflow identity information of the fourth packet, wherein the fourthpacket includes a fourth flow; determining whether the fourth flow is alarge flow; and when the fourth flow is a large flow, forwarding thefourth packet based on a fourth flowlet entry in the flowlet routingtable, wherein the fourth flowlet entry comprises the fourth identifier,or when the fourth flow is not a large flow, forwarding the fourthpacket based on the ECMP routing table.